This notebook will demonstrate the process of using a deep neural network in R to perform image classification.
Necessary libraries:
In [1]:
library(gdata)
library("jpeg")
library("imager")
We use the following function to load the images:
In [2]:
getFileNameAndExtension <- function(file){
ex <- strsplit(basename(file), split="\\.")
ex<-unlist(ex)
return(ex)
}
In [3]:
firstVideoFileName<-'marker.mov'
secondVideoFileName<-'bottle.mov'
outFilePrefix='out'
outFileSuffix='.jpg'
thumbNailFolder<-"Small"
In [4]:
nameAndExtension<-getFileNameAndExtension(firstVideoFileName)
fnames <- paste0(thumbNailFolder,nameAndExtension[[1]],"/","out", 2, ".jpg")
fnames
In [5]:
sampleMarker<-readJPEG(fnames,native=TRUE)
plot(0:1,0:1,type="n",ann=FALSE,axes=FALSE)
rasterImage(sampleMarker,0,0,1,1)
In [6]:
nameAndExtension<-getFileNameAndExtension(secondVideoFileName)
fnames <- paste0(thumbNailFolder,nameAndExtension[[1]],"/","out", 2, ".jpg")
fnames
In [7]:
sampleBottle<-readJPEG(fnames,native=TRUE)
plot(0:1,0:1,type="n",ann=FALSE,axes=FALSE)
rasterImage(sampleBottle,0,0,1,1)
We will use the grayscale intensity of each image as the feature input for a neural network.
In [8]:
dim(sampleMarker)
In [9]:
gs.matrix<-as.matrix(sampleMarker)
dim(gs.matrix)
In [10]:
i.vector<-unmatrix(gs.matrix,byrow=T)
length(i.vector) # unmatrix the gray intensity image as a single column vector, which will be used as feature values to train neural network
Next we label the images with the following function:
In [11]:
loadImagesfromFolderAndLabel <- function(pathName,prefix, clasLabel) {
imageNames<-paste0(prefix,"*.*")
files<-list.files(path=pathName,pattern=imageNames,all.files=T,full.name=T,no..=T)
list_of_images=lapply(files,load.image)
#list_of_images=lapply(files,read.pnm)
#plot(list_of_images[[1]])
n.images<-length(list_of_images)
thmb.matrix<-as.matrix(list_of_images[[1]])
image.vector<-unmatrix(thmb.matrix,byrow=T)
for(ii in 2:n.images)
{
thmb.matrix<-as.matrix(list_of_images[[ii]])
i.vector<-unmatrix(thmb.matrix,byrow=T)
image.vector<-rbind(image.vector,i.vector)
}
image.frame<-data.frame(image.vector)
n.rows<-nrow(image.frame)
class1.label<-rep(clasLabel,n.rows)
image.frame<-cbind(image.frame,class1.label)
rm(list_of_images)
return (image.frame)
}
We then create the input vectors:
In [12]:
nameAndExtension<-getFileNameAndExtension(firstVideoFileName)
folderName<-paste0(thumbNailFolder,nameAndExtension[[1]])
folderName
In [13]:
class1.frame <- loadImagesfromFolderAndLabel(folderName,outFilePrefix,1)
dim(class1.frame)
In [14]:
nameAndExtension<-getFileNameAndExtension(secondVideoFileName)
folderName<-paste0(thumbNailFolder,nameAndExtension[[1]])
folderName
In [15]:
class2.frame <- loadImagesfromFolderAndLabel(folderName,outFilePrefix,-1)
dim(class2.frame)
In [16]:
total.frame<-rbind(class1.frame,class2.frame)
dim(total.frame)
Then we create a training and test set, using 60% of the images for the training set and 40% for the test set:
In [17]:
train.index<-sample(nrow(total.frame),nrow(total.frame)*0.6)
training.set<-total.frame[train.index,]
training.set
In [18]:
test.set<-total.frame[-train.index,]
test.set
Next we train the neural network, using one hidden layer, 20 hidden nodes, and 500 repetitions:
In [19]:
library(neuralnet)
myform <- as.formula(paste('class1.label ~ ',paste(names(training.set[!names(training.set) %in% 'class1.label']), collapse = ' + ')))
oneHiddenLayerClassifier <- neuralnet(myform, training.set, hidden = 20, rep=500, linear.output = FALSE, threshold = 0.01)
summary(oneHiddenLayerClassifier)
Next we make classification predictions:
In [20]:
class.index<-length(test.set)
predictionResults<-compute(oneHiddenLayerClassifier,test.set[,-class.index])
predictionResults$net.result
In [21]:
classifications<-ifelse(predictionResults$net.result>0.5,1,-1)
classifications
In [22]:
table(test.set[,class.index],classifications)
Classification accuracy: 100%
For a performance comparison on the test set, we will train a second deep neural network with two hidden layers with a 5x4 set of hidden nodes in each layer:
In [23]:
myform <- as.formula(paste('class1.label ~ ',paste(names(training.set[!names(training.set) %in% 'class1.label']), collapse = ' + ')))
twoHiddenLayerClassifier <- neuralnet(myform, training.set, hidden = c(5, 4), rep=500, linear.output = FALSE, threshold = 0.01)
summary(twoHiddenLayerClassifier)
In [24]:
class.index<-length(test.set)
predictionResults<-compute(twoHiddenLayerClassifier,test.set[,-class.index])
predictionResults$net.result
In [25]:
classifications<-ifelse(predictionResults$net.result>0.5,1,-1)
classifications
In [26]:
table(test.set[,class.index],classifications)
Classification accuracy: 100%
So, there is no benefit to increasing complexity of the deep neural network for a toy classification problem this small.